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ABSTRACT 


In  an  evolution  from  academic  journal  prestige  ranking 
procedures  used  in  economics,  marketing,  sociology  and 
previous  efforts  in  information  systems  (IS) ,  this  study 
developed  prestige  weights  for  IS  journals  based  on  a 
probability  sample  of  400  MIS  faculty  in  the  United  States  and 
Canada.  The  weights  are  derived  from  subjective  rankings  by 
survey  questionnaire.  In  addition  to  developing  a  master 
schedule  of  journal  prestige  weights,  differences  in  prestige 
weighting  are  examined  according  to  location  of  respondents' 
employing  school,  respondents'  academic  specialties  within  IS, 
types  of  degrees  held  by  the  respondents,  and  years  in  which 
respondents  earned  terminal  degree  at  a  graduate  school  with 
a  national  ranking.  Characteristics  of  respondents  are 
compared  to  nonrespondents  to  test  the  hypothesis  the 
respondents  are  representative  of  the  sample  frame. 
Implications  of  the  procedure  developed  here  are  discussed  in 
terms  of  its  application  to  evaluation  of  research 
productivity  in  DOD  laboratories. 
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I .  INTRODUCTION 


A.  BACKGROUND 

Although  they  are  not  quantified,  subjective  notions  of 
the  differential  prestige  of  academic  journals  are  a  major 
factor  in  decisions  about  hiring,  promotion  and  tenure  for 
information  systems  (MIS)  faculty.  Rather  than  evaluate  a 
faculty  member  by  the  number  of  published  articles,  decision 
makers  are  drawn  to  judge  a  publication  record  in  terms  of 
the  prestige  of  the  particular  journals  in  which  articles 
appear.  This  weighting  process  is  subjective  and 
impressionistic.  Borrowing  from  the  work  of  reference 
disciplines,  especially  sociology  (Glenn,  1971) ,  this  study 
develops  a  schedule  of  prestige  weights  for  IS  journals. 

The  weights  are  based  on  a  survey  of  400  MIS  faculty  listed 
in  the  1992  Directory  of  Management  Information  System 
Faculty. 

The  schedule  of  prestige  weights  is  provided  for  24 
journals  from  MIS  and  related  fields.  We  examined  the 
differences  in  the  weighted  ranking  of  IS  journals  according 
to  the  respondent's  prestige  of  university,  region,  academic 
specialty,  degree  type,  and  degree  date. 

Prestige  of  IS  journals  is  based  on  perceptions  of 
academicians  in  the  MIS  field.  Many  methodologies  (e.g. 
citation  analysis,  ranking,  weighting)  have  been  used  by 
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sociologists  (Glenn,  1971,  and  Shamblin,  1970),  economists 
(Laband,  1985,  and  Medoff ,  1989) ,  and  MIS  professionals 
(Shim  et  al . ,  1987,  and  Ramesh  and  Stohr,  1987)  to  obtain 
journal  rankings. 

B.  OBJECTIVE  AMD  SCOPE 

The  objective  of  this  thesis  is  an  extensive  examination 
of  the  methodologies  used  in  quantifying  the  impressionistic 
value  of  prestige.  A  survey  of  MIS  academicians  will  be 
used  to  determine  a  weighted  ranking  of  IS  journals.  Five 
independent  variables  will  be  used  in  conjunction  with  the 
survey  data  to  determine  if  they  have  any  effect  on  the 
weighted  ranking  or  response  rate. 

With  DoD's  recent  interest  in  re-engineering  business 
processes,  this  study  provides  a  tool  for  measuring 
performance  evaluations.  DoD  has  the  continual  problem  of 
evaluating  a  manager's  performance,  in  many  respects  a 
quantification  of  abstract  qualities.  The  methodology  used 
in  our  study  has  significant  implications  in  terms  of  its 
application  to  evaluations  of  research  productivity  in  DoD 
laboratories  and  schools. 
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II.  LITERATURE  REVIEW 


A.  BRIEF  BACKGROUND 

The  field  of  Management  Information  Systems  (MIS)  has 
grown  in  proportion  to  the  technological  advances  in 
computer  hardware  and  software.  Concurrently,  universities 
have  opened  MIS  departments  and  are  expanding  their  academic 
staff  for  the  purpose  of  teaching  and  doing  research.  As  in 
most  fields,  article  publication  is  an  important  part  of  the 
hiring,  promotion  and  tenure  processes.  A  1980  survey  of 
deans  associated  with  the  American  Association  of  Collegiate 
Schools  of  Business  (AACSB)  indicated  that  publication  was  a 
requirement  for  promotion  and  publication  in  a  journal  of 
merit  has  a  definite  impact  on  hiring  decisions.  This  is 
supported  by  research  done  by  Mabry  and  Sharplin  (1985) . 

They  found  that  the  quality  or  prestige  of  the  publication 
had  an  impact  when  promotions  were  considered. 


B.  REASON  FOR  RANKING  JOURNALS 

Determining  which  journals  are  prestigious  is  important 
in  the  MIS  discipline.  A  list  of  IS  journals,  ranked 
according  to  mean  weight  will  aid  faculty  who  are  in  the 
position  to  make  hiring,  promotion,  and  tenure  decisions. 
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C.  BASIS  FOR  THE  RANKING 

Doke  and  Luke  (1987)  equated  prestige  with  quality. 
Prestige,  being  an  abstract  property,  is  defined  in  the 
Webster's  Unabridged  Dictionary  as  "standing  or  estimation 
in  the  eyes  of  people;  weight  or  credit  in  general  opinion; 
subject  attribute,  often  used  interchangeably  with  quality." 
In  studies  by  Lewis  (1968)  and  Knudsen  and  Vaughan  (1969) , 
prestige  was  considered  different  from  quality,  both  in  the 
way  it  is  measured  and  its  conceptual  rather  than  concrete 
existence . 

Shamblin  (1970)  contends  that  objective  tests  will  give 
a  measure  of  quality  while  subjective  tests  give  a  measure 
of  prestige.  He  believed  that  some  faculty  members  may 
think  a  journal  has  qualitative  value,  but  is  not 
necessarily  prestigious.  Lewis  (1968)  found  a  disparity 
between  objective  and  subjective  rankings.  He  criticized 
prestige  rankings  by  saying  they  are  meaningless  outside  the 
immediate  community.  They  do  not  relate  to  dependent 
variables  of  the  material  world  as  closely  as  objective 
criteria.  On  the  other  hand,  Knudsen  and  Vaughan  (1969) 
suggest  some  relationships  between  the  objective  and 
subjective  results  of  surveys. 

Doke  and  Luke  (1987)  did  a  survey  on  the  quality  of  IS 
journals  among  the  faculty  of  business  schools.  The 
instructions  were  to  rank  the  top  ten  journals  in  order  of 
decreasing  importance.  An  importance/prestige  index  was 
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created  using  a  formula  for  the  number  of  times  a  journal 
was  given  the  same  rank  by  respondents.  This  survey  used  a 
formula  value  for  prestige.  However,  this  is  not  an 
effective  measure  of  prestige,  since  prestige  is  abstract 
and  quality  does  not  necessarily  equate  to  prestige. 

Regardless  of  how  journal  prestige  is  measured,  it 
affects  decisions  about  hiring,  promotion,  and  tenure. 
According  to  Gordon  and  Purvis  (1991) ,  the  common  goal  of 
determining  a  ranking  of  journals  in  any  of  the  disciplines 
was  to  provide  university  review  committees  with  one  fairly 
objective  measure  of  "research  excellence"  for  use  in 
promotion  and  tenure  decisions. 

1.  Sociology 

Studies  in  the  field  of  sociology  provided  varying 
methodologies  for  gathering  information  to  ascertain  quality 
or  a  ranking  for  sociological  journals.  The  journals  were 
ranked  by  a  variety  of  methods.  Citation  analysis  (the 
number  of  times  a  journal  is  cited)  was  the  most  common 
method  of  determining  the  journals'  quality.  This  method 
does  not  prove  to  be  the  best  for  the  following  reasons: 
(l)the  size  of  the  journal  will  determine  the  number  of 
potentially  citable  items;  (2) a  longer  established  journal 
will  have  more  citations  than  recently  established  journals; 
(3) journal  circulation,  dissemination  of  reprints, 
availability  in  library  collections  and  coverage  by 
secondary  indexing  and  abstracting  services  will  influence 


citation  patterns;  (4) a  few  articles  that  are  highly  c  ted 
may  distort  the  citation  patterns  for  a  journal;  (5) journal 
article  length  and  number  of  references  may  influence 
citation  patterns;  (6) journal  prestige  differentiation  will 
influence  citation  patterns  ar.d  (7)  the  reputation  of  authors 
and  the  controversiality  of  subject  matter  published  in  the 
journal  will  influence  citations  patterns  and  a  growth  in 
the  literature  (changes  in  the  number  of  journals  or 
articles  published)  will  influence  citation  patterns. 

Glenn  (1971)  evaluated  the  journals  for  prestige 
using  a  list  of  journals  and  arbitrarily  assigned  a 
benchmark  weight  to  one  of  the  journals.  A  benchmark  weight 
of  ten  was  used  to  keep  a  relative  frame  of  reference  on  the 
numbers  used  in  the  weighting.  For  example,  a  respondent  in 
the  sample,  who  liked  a  journal  as  much  as  another 
respondent  could  use  a  wider  range  of  numbers  (1  to  100 
instead  of  1  to  10)  to  express  his  or  her  opinion  even 
though  the  journal  was  equally  liked  by  both  respondents. 

By  assigning  a  benchmark  weight,  the  two  respondents  will  be 
forced  to  weight  all  the  journals  in  accordance  with  the 
pre-assigned  weighted  journal. 

Glenn  (1971)  sent  the  survey  to  faculty  members 
randomly  drawn  from  the  1969  ASA  Guide  to  Graduate 
Departments  of  Sociology.  This  sample  was  representative  of 
senior  faculty  who  would  be  involved  in  the  hiring, 
promotion,  and  tenure  decision  process,  as  well  as  the 
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junior  faculty  who  would  be  using  the  journals  for  research. 
The  survey  yielded  a  weighted  prestige  ranking  of  the 
sociology  journals.  Glenn  (1971)  believed  that  any  sampling 
biases  that  might  have  occurred  in  the  faculty  due  to 
geographical  region,  prestige  of  university,  differing 
degree  types,  and  age  groups  was  addressed  by  using  a  random 
sample . 

2 .  Economics 

The  literature  on  journal's  prestige  in  economics 
falls  into  two  categories:  (a) those  dealing  with  the 
publishing  performance  of  economists  and  their  departments 
and  (b) those  concerned  with  the  prestige  value  of  journals 
as  a  measure  of  research  performance.  Exploring  the  methods 
of  obtaining  the  prestige  value  was  useful  in  determining 
which  methods  were  to  be  used  in  ranking  journals  within  a 
specific  discipline.  (Medoff,  1989;  Gibbons  and  Fish,  1991) 

Citation  frequency  was  commonly  used  to  determine 
rankings  of  finance  and  economic  journals.  These  citation 
frequencies  were  multiplied  by  factors  based  on  previous 
studies  using  a  subjective  rank  ordering  of  scholarly 
journals  in  the  discipline  based  on  their  perceived  quality 
by  readers  (Ellis  and  Durden,  1991) .  As  an  example,  Mabry 
and  Sharplin  (1985)  based  their  study  on  the  relative 
importance  of  journals  used  in  finance  research  on  a 
previous  survey  by  Coe  and  Weinstock  (1983) .  Coe  and 
Weinstock  (1983)  surveyed  department  chairs  to  seek  a 
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subjective  ranking  of  finance  journals.  However,  there  were 
some  flaws  in  the  survey;  it  was  short  and  had  an  incomplete 
list  of  journals.  It  was  left  to  the  respondents  to  fill  in 
additional  journals  that  they  deemed  as  important. 

According  to  Mabry  and  Sharplin  (1985) ,  "either  no  initial 
list  at  all  or  an  extensive  one  should  be  provided  to  avoid 
bias  toward  listed  journals."  Furthermore,  "while  the 
impressions  of  journal  quality  by  department  chairs  are 
useful  in  certain  contexts,  academic  administrators 
represent  a  small,  specialized  group  and  may  be  poor 
surrogates  for  all  academic  researchers." 

Even  though  survey  methodologies  may  not  always  be 
the  most  appropriate  measure  of  journal  quality  due  to  the 
bias  imposed  by  the  design  of  the  questionnaire  or  the 
groups  selected  to  judge  the  journals,  they  still  provide 
the  subjective  perceptions  of  significant  faculty  with 
respect  to  the  relative  quality  of  academic  journals. 
"However,  the  most  objective  measure  for  addressing  journal 
quality  as  most  scholars  perceive  it  is  the  citation 
frequency  of  published  studies."  (Mabry  and  Sharplin,  1985) 
Citation  frequency  is  an  objective  technique  for  evaluating 
the  impact  of  scholarly  research.  It  has  been  used  more 
frequently  than  subjective  survey  techniques  (Mabry  and 
Sharplin,  1985) . 

A  study  by  Ellis  and  Durden  (1991)  identifies 
citation  frequency  as  one  of  the  two  most  important  factors 
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that  influence  an  economist's  perceptions  of  journal 
quality.  A  journal's  quality  can  reflect  the  impact  its 
articles  may  have  disseminating  knowledge  within  the 
discipline  (impact  measured  by  citation  frequency;  Liebowitz 
and  Palmer,  1984) .  However,  a  journal  can  still  maintain  a 
certain  level  of  prestige  among  colleagues  in  the  profession 
well  after  its  citation  frequency  declines. 

Economists  have  mixed  feelings  regarding  the 
measurement  of  research  quality.  Ellis  and  Durden  (1991) 
and  Liebowitz  and  Palmer  (1984)  believe  strongly  in  citation 
frequency.  Others,  such  as  Medoff  (1989) ,  criticize 
citation  counts  as  a  measure  of  research  quality,  especially 
of  individual  economists.  First,  counting  pages  or  articles 
in  journals  ignores  the  contribution  that  books  may  make  to 
economics.  Second,  some  articles  may  be  published  in  well- 
known  journals  but  may  not  be  used  for  research  or  worse 
yet,  even  read.  "Publication  in  a  major  journal  does  not 
necessarily  measure  the  impact  of  the  work,  its  creative 
insights,  or  its  seminal  contribution,"  Medoff  (1989) 
explains.  Therefore,  articles  in  well-known  journals  are  an 
imperfect  measure  of  quality.  And  third,  there  is  always 
the  possibility  of  bias  in  quantitative  publication  ranking. 
Laband  (1985)  found  that  on  average  over  two  extra  pages  are 
falsely  credited  to  authors  affiliated  with  the  same  school 
as  the  editor  of  a  journal. 
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Hawkins  et  al.  (1973)  explored  the  notion  of 
implicit  prestige  ranking  attached  to  a  journal.  Their 
initial  questionnaire  was  sent  to  160  academic  economists 
which  were  stratified  into  four  groups  of  40  each.  The 
groups  were:  (1) deans  and  department  chairs  at  large 
research- oriented  universities;  (2) faculty  members  at  such 
universities;  (3) individuals  with  major  administrative 
responsibilities  at  smaller  teaching -oriented  colleges;  and 
(4) faculty  members  at  such  colleges.  The  sample  was  also 
stratified  by  age  group  and  academic  department. 

Out  of  the  initial  160  questionnaires  sent,  111  were 
returned.  The  statistical  information  was  sent  back  to  the 
respondents  (with  a  reminder  of  how  they  ranked  the  journals 
the  first  time)  in  a  second  Delphi  wave.  Ninety- two  of 
those  were  returned.  In  the  second  survey,  eighty  percent 
of  respondents  changed  at  least  one  point  score. 

The  study  also  reported  a  familiarity  index  because 
the  percentage  of  respondents  who  reacted  to  each  particular 
journal  were  to  give  a  point  score  only  if  they  were 
familiar  with  a  journal.  An  alternative  prestige  ranking 
based  on  the  mean  point  score  times  the  familiarity  index 
was  shown,  this  gave  an  equal  weight  to  perceived  quality 
and  presumed  readership  in  establishing  the  prestige  of  a 
given  journal. 

The  researchers  also  sought  to  reveal  the  prejudice 
that  economists  have  toward  general,  well-known  journals. 
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They  included  two  fictitious  journal  names  to  test  the 
reliability  of  their  analysis.  The  fictitious  Journal  of 
Economic  and  Statistical  Theory  (J.E.S.T),  ranked  in  the  top 
third  of  all  journals  surveyed  (twenty- fourth  out  of  87) . 

The  other  fictitious  journal,  entitled  Regional  Studies  and 
Economic  Change  (R.S.E.C. ) --obviously  a  more  applied  and 
specialized  journal,  but  equally  nonexistent- -ranked  in  the 
bottom  third.  (Hawkins  et  al.,  1973)  This  demonstrates  an 
ego  attribute  in  journal  ranking.  This  is  that  people  do 
not  like  to  admit  their  own  ignorance  or  are  careless  when 
reviewing  the  titles,  merely  marking  off  a  journal  name  if 
it  sounds  familiar  or  scholarly. 

Gordon  and  Purvis  (1991)  combined  research  from  a 
variety  of  disciplines  to  measure  research  performance  in 
industrial  relations.  The  disciplines  include  economics, 
industrial  psychology,  organizational  behavior,  and 
sociology.  The  study  discussed  three  ways  in  which  journal 
quality  may  be  measured.  First,  by  surveying  a 
representative  sample  of  members  in  the  discipline  to 
construct  a  hierarchy  of  journals.  Second,  by  using 
citation  indicators  to  rank  journals.  And  third,  consulting 
p^ceptance  rates  of  articles  into  journals.  Cabell  (1988) 
reported  the  acceptance  rates  for  hundreds  of  journals  in 
industrial  relations.  One  should  keep  in  mind  that 
"although  acceptance  rates  are  a  fairly  stable  property  of 
journals,  they  may  not  correlate  significantly  with  prestige 
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ratings."  (Hargens,  1988)  After  reviewing  the  three  ways  to 
measure  quality,  we  concluded  that  surveying  a 
representative  sample  is  the  best  course  for  constructing  a 
hierarchy  of  journals. 

3.  Management  Information  Systems  (MIS) 

The  number  of  IS  journals  used  in  the  discipline  is 
growing  and  the  number  of  articles  referring  to  MIS  matters 
in  non- IS  specific  journals  is  also  increasing  (Hamilton  and 
Ives,  1982,  Ramesh  and  Stohr,  1987) .  Business  journals 
include  MIS  topics  even  though  they  are  not  solely  devoted 
to  MIS.  Shim  et  al.  (1987)  and  Davis  and  Fry  (1986)  did 
studies  which  ranked  IS  journals,  however,  their  studies 
used  only  five  and  twelve  journals,  respectively. 

Other  methods  of  ranking  have  been  attempted.  Koong 
and  Weistroffer  (1989)  ranked  the  top  15  MIS  journals  from  a 
list  of  70  journals.  They  asked  colleagues  to  choose  the 
three  best  journals  for  reading  and  the  three  best  journals 
for  publishing.  This  was  not  necessarily  an  indication  of 
quality.  If  a  certain  journal  has  a  higher  article 
acceptance  rate  than  another,  then  more  colleagues  will 
respond  favorably  to  that  particular  journal  for  publishing, 
it  does  not  mean  they  have  a  high  opinion  of  the  journal  for 
reading  purposes. 

Nord  and  Nord  (1991)  used  the  results  of  three  other 
studies  to  pick  the  top  IS  journals.  IS  journals  common  in 
the  top  15  of  all  three  studies  were  picked  as  the  top  five 
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IS  journals.  This  procedure  gave  the  MIS  field  only  five 
top  journals.  But  a  larger  number  of  journals  are  currently 
being  used  and  need  to  be  recognized  in  the  MIS  field.  As 
mentioned  previously,  Koong  and  Weistroffer  (1989)  listed 
over  70  IS -related  journals.  Therefore,  limiting  the  number 
of  top  journals  to  five  does  not  give  the  selection  or 
variation  needed  for  studying  the  growing  MIS  field. 

Although  the  70  journals  were  not  dedicated  solely  to  MIS 
issues,  they  still  provided  information  to  MIS  colleagues 
as  well  as  those  affiliated  with  related  disciplines. 

Hamilton  and  Ives  (1983)  did  a  six  measure  journal 
stratification  resulting  in  four  strata  of  journals  using  a 
composite  indicator  of  journal  importance.  The  journals 
were  split  into  four  disciplines;  MIS,  Computer  Science, 
Management  Science,  and  Management.  Since  the  MIS 
discipline  is  interwoven  in  other  disciplines,  the 
distinction  can  not  be  made  between  the  material  within  the 
journals  of  each  discipline.  Because  the  articles  are 
interchangeable,  not  belonging  exclusively  to  one 
discipline,  it  is  not  imperative  to  have  separate  categories 
for  ranking  purposes. 

Nonetheless,  in  the  Hamilton  and  ~ves  (1983)  study, 
the  surveys  went  to  a  senior  group  of  "knowledgeable  and 
recognized  experts"  in  the  academic  MIS  community.  It  was 
believed  they  could  provide  a  more  informed  judgement  about 
journal  contributions  than  junior  faculty.  However,  many 
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less-known  MIS  faculty  are  involved  in  promotion  and  tenure 
decisions.  Also,  junior  faculty  are  active  in  publishing 
(often  more  than  senior  faculty)  and  are  more  anxious  about 
publishing  in  prestigious  journals.  Senior  faculty  either 
have  stopped  publishing  or  can  publish  with  ease  in  any 
journal  because  they  are  well-known  faculty  statesmen.  They 
are  less  sensitive  to  the  current  prestige  of  journals. 

They  would  respect  the  opinion  of  their  honored  colleagues, 
but  probably  do  not  know  what  those  opinions  are.  They  have 
their  own  opinions  about  journal  prestige  which  would  be  an 
influential  factor  in  their  decision  processes. 

Ramesh  and  Stohr  (1987)  used  two  separate  surveys  to 
study  journal  prestige.  The  first  was  sent  to  journal 
editors  and  conference  chairs  to  understand  the  refereeing 
and  acceptance  process.  The  second  was  sent  to  200 
researchers  in  computer  science  (100  obtained  from  a  list  of 
department  chairs  at  major  United  States  and  Canadian 
universities,  the  other  100  eminent  in  computer  science 
research  areas)  and  200  researchers  in  MIS  (200  obtained 
from  the  1987  McGraw-Hill  Directory  of  Information  Systems 
Faculty)  .  Their  aim  was  to  measure  the  perceived  research 
quality  of  33  journals  and  14  conference  proceedings.  The 
study  shows,  once  again,  how  such  perceptions  influence 
hiring,  promotion  and  tenure  decisions. 

Although  a  number  of  publications  were  related  to 
the  MIS  field,  some  were  not  necessarily  core  IS  journals. 
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Ramesh  and  Stohr  (1987)  also  noted  the  inherent  problems  of 
opinion  surveys;  they  are  void  of  objective  facts  and  suffer 
biases,  while  measuring  subjective  perceptions.  Computer 
science  respondents  returned  forty- two  percent  of  the 
surveys  and  MIS  respondents  returned  sixty  percent  of  the 
surveys.  Most  of  the  respondents  were  full  professors.  A 
familiarity  question  ranging  from  "don't  know"  to  "know 
well"  was  included  in  the  survey.  This  was  an  attitudinal 
question  about  behavior.  A  research  quality  question 
ranging  from  "poor"  to  "excellent"  was  also  part  of  the 
survey.  Those  respondents  who  claimed  no  knowledge  of  a 
publication  but  gave  it  a  quality  rating  were  excluded  from 
the  analysis.  In  a  comparison  with  Hamilton  and  Ives  (1983) 
and  Shim  et  al.  (1986)  studies  of  the  top  ten  MIS  journals, 
Ramesh  and  Stohr' s  (1987)  findings  were  in  agreement  with 
eight  journals  appearing  in  both  studies.  Even  though 
Ramesh  and  Stohr  (1987)  provided  information  about  the 
perceived  quality  of  MIS  journals,  they  noted  that  "the 
relative  rankings  of  the  publications  . . .  indicate  an 
academic  field  that  is  still  in  its  formative  stages." 


D.  CONCLUSION 

In  every  discipline  there  are  two  distinct  career  paths 
to  follow.  One  pursues  the  discipline  as  a  professional  or 
as  an  academician.  Both  careers  need  the  most  current 
information  available  in  the  field,  but  may  use  publications 
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in  a  different  manner  (Babchuk  and  Bates,  1962) .  This  study 
showed  that  professionals  referred  less  often  to  journals 
and  publications  than  do  academicians.  Therefore,  IS 
academicians  are  best  suited  to  weight  IS  journals. 

In  addition,  a  study  modeled  after  Glenn's  (1971) 
sociological  study  on  perceived  journal  prestige  by 
weighting  has  not  been  done  in  the  MIS  field.  Such  a  study 
would  provide  reference  by  which  the  vita  of  candidates  for 
hiring,  promotion  and  tenure  could  be  judged. 
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Ill .METHODOLOGY 


A  methodology  influenced  by  the  work  of  Glenn,  (1971) 
was  used  to  obtain  a  ranking  of  IS  journals.  It  identifies 
prestigious  IS  journals  through  a  survey  of  MIS 
academicians.  The  findings  in  this  study  are  based  solely 
on  the  respondents  perceptions  of  IS  journals. 

A.  TEST  SURVEY  AND  QUESTIONNAIRE  SELECTION 

The  survey  methodology  was  narrowed  down  to  two 
different  forms:  (a) FORM  1  included  a  cover  letter  with 
instructions  to  rank  the  top  ten  journals  from  the  list  of 
IS  core  journals.  The  journals  were  to  be  ranked  (one 
through  ten)  in  accordance  with  each  colleague's  judgement 
and  (b) Form  2  included  a  cover  letter  with  instructions  to 
assign  weights  to  the  list  of  IS  core  journals. 

A  test  survey  of  each  method  was  conducted  among  members 
of  the  MIS  staff  at  the  Naval  Postgraduate  School  (NPS) .  A 
list  of  20  IS  core  journals  was  obtained  by  using  a 
consensus  of  journal  names  from  Hamilton  and  Ives  (1983), 
Davis  (1980)  and  Vogel  and  Wetherbe  (1984) .  The  twelve 
member  staff  first  received  FORM  1  (Appendix  A)  and  three 
weeks  later  received  FORM  2  (Appendix  B) . 

On  FORM  2,  Management  Information  Systems  Quarterly 
(MISQ)  was  selected  as  a  standard  of  reference,  and  was 
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given  a  weight  of  10.  Other  journals  were  to  be  weighted 
accordingly. 

Comments  associated  with  FORM  1  (the  ranking)  were:  One 
staff  member  noted  a  need  for  more  categories  of  journals 
(e.g.  Research,  Theoretical,  Application,  Computer  Science, 
and  Management) .  Another  found  the  familiarity  issue  a 
problem.  Respondents  were  uncertain  how  to  respond  if  a 
journal  was  unfamiliar;  give  it  a  low  score  or  none  at  all. 
There  was  also  a  sense  of  curiosity  associated  with  the  mix 
of  scholarly  and  trade  publications. 

FORM  2  (the  weighting)  comments  were:  One  staff  member 
said  that  it  would  have  been  easier  if  we  stated  the  point 
of  view  of  which  the  grading  was  based  (e.g.  readership, 
authors) .  Weighting  the  journals  using  MISQ  as  a  reference 
seem  to  confound  some  staff  members.  They  felt 
uncomfortable  assigning  weights  as  two  times  or  one  half  as 
important  as  MISQ  which  was  given  a  ten.  One  concern  which 
was  universal  to  both  surveys  was  that  the  list  of  IS  core 
journals  was  not  inclusive.  The  following  journals  were 
noted  as  missing  from  the  list:  Decision  Support  Systems ,- 
ORSA  Journal  on  Computing;  ACM  Transactions;  IEEE 
Transactions;  Corporate  Computing;  CIO  Journal;  CIO 
Magazine.  All  but  the  last  three  journals  were  added  to  the 
list  of  IS  core  journals.  The  last  three  were  mentioned  by 
only  one  staff  member. 
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After  reviewing  the  responses  to  the  test  survey,  the 
decision  was  made  to  select  FORM  2  for  use  in  the  actual 
survey.  It  seemed  to  elicit  an  overall  positive  response 
from  the  MIS  staff.  Besides  the  addition  of  four  more 
journals  to  the  list,  minor  adjustments  were  made  to  the 
cover  letter  to  make  it  more  formal  and  suitable  for  a 
general  mailing. 

Four  hundred  copies  of  the  cover  letter  (Appendix  C) 
were  made  using  letterhead  to  make  the  survey  appear 
official  and  400  sheets  of  blue  pastel  paper  were  used  for 
the  new  list  of  IS  core  journals. 


B .  THE  SAMPLE 

In  order  to  get  a  representative  sample  of  "MIS 
knowledgeable"  people  who  would  be  generally  active  and 
experienced  in  the  world  of  academia,  a  random  sample  was 
drawn  from  the  1992  McGraw-Hill  Directory  of  Information 
Systems  Faculty.  Four  hundred  MIS  faculty  names  (out  of 
1800)  were  chosen  to  represent  about  20  percent  of  the  total 
population.  To  achieve  a  random  sampling,  the  fourth  name 
and  fifth  alternately  were  selected  from  the  MIS  directory. 
The  starting  point  was  the  seventh  name  (which  was  chosen  by 
tossing  dice) . 

The  respondent's  and  non- respondent's  prestige  of 
university,  region,  academic  specialty,  degree  type,  and 
degree  date  would  be  used  in  the  analysis  of  our  data. 
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Therefore,  the  names  of  all  chosen  to  receive  surveys  were 
entered  in  a  database  which  contained  addresses, 
universities,  regions,  academic  specialties,  degree  types, 
and  degree  dates. 

C.  CONDUCT  OP  STUDY 

To  obtain  the  maximum  results  for  a  weighted  ranking  IS 
core  journals,  a  decision  was  made  to  send  surveys  to  MIS 
faculty  members  of  universities  throughout  the  country. 
Prestige,  as  defined  earlier,  is  an  abstract,  subjective 
quality  used  in  the  evaluation  of  people's  work.  There  is 
no  concrete  measure  of  prestige  but  it  does  play  an 
important  role  in  decision  processes.  A  subjective  prestige 
ranking  of  journals  would  be  the  best  guide  for  faculty 
making  hiring,  promotion,  and  tenure  decisions. 

The  400  surveys  were  mailed  on  10  April,  1993,  early 
enough  in  the  spring  quarter  to  arrive  before  faculty  began 
their  summer  break.  Completed  forms  began  to  arrive  within 
one  week.  Responses  continued  to  trickle  in  for  three  more 
weeks,  then  there  was  a  serious  drop  in  returns  and  only  ten 
more  arrived  over  the  next  six  weeks.  Of  the  400 
questionnaires  mailed  out,  nine  were  returned  with 
absolutely  no  weights  given  and  various  excuses  for  the  lack 
of  response  (the  most  frequent  being  that  the  respondent  was 
not  familiar  with  any  journals  or  with  MISQ)  and  four  more 
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were  returned  by  the  postal  service  and  marked  as 
undeliverable . 

One  hundred  sixty-eight  were  returned  with  IS  journals 
weighted.  Therefore,  396  usable  surveys  were  sent  and  42 
percent  were  returned.  This  is  comparable  to  other  surveys 
which  had  response  rates  of  37.8  percent  (Hamilton  and  Ives, 
1983) ,  31  percent  (Doke  and  Luke,  1987)  and  almost  50 
percent  (Glenn,  1971) . 

D.  VALIDATION 

The  complete  database  was  printed  out  and  validated 
manually  by  a  two -person  check.  We  compared  the  results  of 
each  respondent's  completed  survey  to  their  individual  entry 
in  the  database.  One  numeric  typing  error  was  found  and  was 
changed  before  the  data  was  analyzed. 

E.  ANALYSIS  OF  RESPONDENTS  VERSUS  NON- RESPONDENTS 

An  analysis  was  conducted  based  on  a  statistical 
evaluation  of  the  survey  data  using  SPSS  release  4.1  for  IBM 
VM/CMS.  In  addition  to  our  overall  IS  journal  weighted 
ranking,  we  conducted  analyses  on  survey  respondents  versus 
non- respondents  over  several  dimensions.  These  include 
prestige  of  university,  region,  academic  specialty,  degree 
type,  and  degree  date.  We  were  interested  in  studying 
whether  any  of  the  variables  would  affect  response  rate. 
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1.  Prestige  of  University  vs.  Response  Rates 


Table  1  shows  the  number  of  respondents  and  non¬ 
respondents  and  whether  they  obtained  their  degree  from  a 
nationally  ranked  university  (Lord,  1993) . 


TABLE  1 

RESPONDENTS  AND  NON -RESPONDENTS  BY  PRESTIGE  07  UNIVERSITY 


Respond 

Ranked 

Not  Ranked 

Total 

No 

95 

138 

233 

Yes 

67 

101 

168 

total 

162 

239 

401 

Ho:  No  difference  in  the  responses  of  graduates  of 
ranked  universities  and  non-ranked  universities. 

Ha:  Whether  a  person  responded  to  the  survey  and  the 
ranking  of  the  university  where  they  received  their  degree 
are  statistically  dependent. 


Chi  square 

Df 

Significance 

Phi 

.0322 

1 

.858 

.009 

In  this  case  we  fail  to  reject  the  null  hypothesis. 
We  conclude  that  survey  response  and  the  ranking  of  a 
university  where  a  respondent  received  his  or  her  degree  are 
statistically  independent. 

2.  Regions  vs.  Response  Rates 

Region  is  defined  as  one  of  four  geographical  areas 


22 


into  which  the  United  States  was  split.  Table  1  gives  the 
breakdown  of  states  into  their  respective  region. 


TABLE  2 

REGIONS  OF  TEE  UNITED  STATES 


|  East 

Midwest 

South 

West 

CT 

DC 

DE 

IA 

IL 

IN 

AL 

AR 

FL 

AK 

AZ 

CA 

MA 

MD 

ME 

MI 

MN 

MO 

GA 

KY 

LA 

CO 

HI 

ID 

NH 

NJ 

NY 

ND 

NE 

OH 

MS 

NC 

SC 

MT 

NM 

NV 

PA 

RI 

VA 

OK 

SD 

WI 

TN 

TX 

WV 

OR 

UT 

WA 

VT 

WY 

The  geographical  distributions  of  respondents  and 
non- respondents  can  be  seen  in  Table  2. 


TABLE  3 

RESPONDENTS  AND  NON- RESPONDENTS  BY  REGION 


Respond 

East 

Midwest 

South 

West 

Total 

No 

56 

70 

57 

49 

232 

Yes 

40 

43 

53 

32 

168 

Total 

96 

113 

110 

81 

400 

Ho:  No  difference  in  the  responses  according  to  the 
region  of  residency. 

Ha:  Whether  a  person  responds  to  the  survey  and  what 
region  of  the  country  they  are  from  are  statistically 
dependent . 
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In  this  case  we  fail  to  reject  the  null  hypothesis. 
We  conclude  that  survey  response  rate  and  what  region  the 
respondent  is  from  are  statistically  independent  of  each 
other. 

The  geographical  information  shows  that  the  sample 
is  evenly  distributed  across  the  four  regions  of  the  United 
States . 

3.  Academic  Specialties  vs.  Response  Rates 

Table  4  shows  the  number  of  respondents  and  non- 
respondents  and  their  academic  specialties  (Appendix  D) . 

The  respondent's  academic  specialty  is  the  area  in  which 
they  work  as  delineated  in  the  1992  McGraw-Hill  Directory  of 
Information  Systems  Faculty. 


TABLE  4 

RESPONDENTS  AND  NON- RESPONDENTS  BY  ACADEMIC  SPECIALTY 


Respond 

8.1 

00 

to 

8.3 

8.4 

8.5 

Unk 

Total 

No 

84 

66 

53 

4 

3 

23 

233 

Yes 

59 

53 

46 

3 

1 

6 

168 

Total 

143 

119 

99 

7 

4 

29 

401 

Ho:  No  difference  between  responses  to  the  survey 
according  to  academic  specialty. 
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Ha:  Whether  a  person  responds  to  the  survey  and  their 


academic  specialty  are  statistically  dependent. 


Chi  Square 

Df 

Significance 

Cramer's  V  1 

7.043 

5 

.217 

.133  1 

In  this  case  we  fail  to  reject  the  null  hypothesis. 
We  conclude  that  survey  response  rate  and  academic  specialty 
are  statistically  independent  of  each  other. 

4.  Degree  Types  vs.  Response  Rates 

The  degree  types  were  split  into  four  disciplines; 
IS,  business,  social  sciences,  and  mathematics  and  physical 
sciences.  The  following  table  shows  the  breakdown  of  degree 
types . 


TABLE  5 
DEGREE  TYPES 


Information 

Systems 

Business 

Social 

Sciences 

Math  &  Phys 
Sciences 

Sys  Design 
Decision  Sci 
Micro  App 

MIS  and  IS 

Art  Intel 

Admin 

Statistics 

Accounting 

Ops  Research 

Management 

Finance 

Education 

Org  Behavior 

Economics 

Psychology 

Sociology 

Philosophy 

Engineering 
System  Sci 

Comm  Sci 

Table  6  shows  the  degree  type  and  the  number  of 
respondents  and  non- respondents .  The  four  unknowns  are 
people  whose  names  and  university  were  in  the  MIS  directory 
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but  degree  type  was  not  listed,  and  one  anonymous  respondent 
who  sent  in  a  weighting  of  the  journals.  An  extra  record 
with  no  name  or  biographical  information  (only  the 
weightings)  was  created  for  the  anonymous  respondent. 
Therefore,  the  sample  reads  401,  vice  400. 


TABLE  6 

RESPONDENTS  AND  NON- RESPONDENTS  BY  DEGREE  TYPE 


Respond 

IS 

Business 

Soc  Sci 

Math  & 
Phys  Sci 

Unknown 

Total 

No 

95 

55 

43 

21 

19 

233 

Yes 

75 

57 

17 

15 

4 

168 

Total 

170 

112 

60 

36 

23 

401 

Ho:  No  difference  in  response  to  survey  according  to 
degree  type. 


Ha:  Whether  a  person  responds  to  the  survey  and  their 
degree  type  are  statistically  dependent . 


Chi  Square 

Df 

Significance 

-  -  .... - 

Cramer's  V 

14.277 

4 

.006 

.189 

In  this  case  we  reject  the  null  hypothesis. 
Therefore,  we  can  conclude  that  a  relationship  exists 
between  the  survey  response  rate  and  the  respondent's  degree 
type. 

5.  Degree  Dates  vs.  Response  Rates 

Table  7  shows  the  number  of  respondents  and  non¬ 
respondents  and  their  degree  dates  (by  the  decade) .  The 
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e  MIS  directory.  The 


Ho:  No  difference  in  response  to  the  survey  according  to 
degree  date. 

Ha:  Whether  a  person  responds  to  the  survey  and  the  date 
they  received  their  degree  are  statistically  dependent. 


Chi  Square 

Df 

Significance 

Cramer's  V 

2.437 

4 

.656 

.080 

In  this  case  we  fail  to  reject  the  null  hypothesis. 


We  conclude  that  survey  response  rate  and  degree  date  are 
statistically  independent  of  each  other. 

Except  for  type  of  degree,  we  found  no  statistically 
significant  differences  between  respondents  and  non¬ 
respondents.  Therefore,  the  respondents  to  this  survey  may 
be  characterized  as  representative  of  the  population  of  MIS 
academicians  as  they  are  characterized  by  a  random  sample. 
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IV.  FINDINGS 


A.  IS  JOURNAL  WEIGHTED  RANKING 

As  with  Glenn's  (1971)  study,  the  IS  journals  were 
weighted  and  ranked  by  prestige  according  to  the 
respondent's  perceptions  (Table  8).  Because  the  weights 
assigned  to  the  IS  journals  were  identified  as  interval 
level  data,  the  mean  and  standard  deviation  were  identified 
as  the  appropriate  measure  of  central  tendency  and 
dispersion,  respectively. 

1.  Intensity  end  Extensity  of  Prestige 

The  overall  IS  journal  ranking  was  also  used  to 
study  the  differences  between  the  intensity  and  extensity  of 
prestige,  Glenn  (1971)  states,  "the  mean  weight  is 
essentially  an  indicator  of  the  intensity  of  prestige, 
whereas  the  number  of  respondents  who  assigned  weights 
(Table  8,  N  column)  is  a  rough  indicator  of  how  well  the 
journal  was  known  among  the  respondents,  or  the  extensity  of 
the  journal's  prestige."  The  number  of  respondents  who 
assigned  weights  can  not  be  considered  a  precise  measure  of 
the  extensity  of  prestige,  since  some  respondents  may  have 
thought  a  journal  prestigious  but  didn't  feel  qualified  to 
make  a  precise  judgement. 

We  tested  the  precision  of  our  measure  of  extensity. 
We  formulated  a  figure  of  total  prestige  by  multiplying  the 
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intensity  by  the  extensity  (Table  8,  Total  column)  to  see  if 
the  order  of  the  IS  journal  ranking  would  be  affected. 

As  can  be  seen  in  Table  8,  ORSA  Journal  on  Computing 
received  a  higher  mean  weight  (9.2)  than  Data  Base  (6.88) 
but  the  latter  was  known  and  evaluated  by  more  respondents. 
The  journals  can  be  ordered  by  either  criteria  (mean  weight 
or  number  of  respondents  who  assigned  weights)  but  there  is 
no  sound  basis  to  conclude  that  neither  ORSA  Journal  on 
Computing  or  Data  Base  had  more  overall  prestige.  On  the 
other  hand,  if  two  of  the  IS  journals  had  similar  mean 
weights,  than  total  prestige  could  be  determined.  For 
instance.  Information  and  Management  (7.53)  and  Omega  (7.5) 
had  similar  mean  weights.  The  former  was  rated  by  128 
respondents  and  the  latter  by  80 — an  indication  of  the 
extensity  of  the  prestige  of  Information  and  Management. 

Table  8  compares  the  weighted  rank  order  of  IS 
Journals  (Mean  column)  and  the  rank  order  when  the  weighted 
ranking  is  multiplied  by  the  N  of  respondents  (Order 
column).  A  Spearman's  Rank  order  correlation  shows  a 
moderately  strong  association  between  the  two  rank  orders 
(rho  =  .76).  We  therefore  concluded  that  the  weighted  rank 
order  was  not  a  biased  indicator  of  IS  journal  prestige  when 
accounting  for  extensity. 
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TABLE  8 


OVERALL  18  JOURNAL  WEIGHTING 


M 

SMDev 

Total 

(Mar 

UmogOTunt  Sdmof 

12.67 

141 

6.91 

1772.37 

1 

Information  Systems  Rusotfi 

1143 

124 

446 

.1417.32 

5 

Commurtcaicni  ol  fra  ACM 

11.06 

162 

461 

1686.66 

2 

CEE  Transactions 

10S6 

122 

6.64 

1327.36 

7 

AOnfciMr— v  Science  Quarterly 

10.56 

101 

6.79 

1068.58 

14 

ACM  Transaction* 

10.1 

126 

6.06 

1302.9 

8 

IteMQtmsnt  Information  Systems  Quarterly 

10.01 

154 

1.06 

164144 

3 

ACM  Computing  Surveys 

9.97 

136 

6.41 

137646 

6 

nwvwu  dussisss  nwNw 

8.9 

147 

6.15 

1466.3 

4 

Academy  ol  Management  Journal 

6.62 

113 

447 

1067.06 

13 

ORSA  Journal  on  Compiling 

9.2 

74 

6.97 

660.8 

22 

Dsdtion  Sctencos 

9.14 

137 

6.11 

1262.18 

10 

IEEE  Computer 

9.04 

123 

4.22 

111142 

12 

Journal  tf  Management  Monnallon  Systems 

8.88 

146 

3.06 

1302.1 

9 

Sloan  Managamsnt  Review 

6.9 

133 

4.32 

1183.7 

11 

Accounting  Review 

IM 

66 

6.16 

737.88 

20 

Decision  St^jport  Systems 

6.4 

114 

4.62 

657.6 

17 

tnttirtootit 

7.81 

127 

438 

100457 

16 

Information  and  Managamant 

7JB3 

126 

3.02 

963.84 

16 

Omega 

7.6 

60 

2.87 

600 

24 

DataBase 

6J8 

136 

3.64 

928.8 

18 

Journal  ol  Systems  Managamant 

6.56 

127 

3.4 

833.12 

19 

EOP  Analyzer 

5M 

104 

6.7 

612.56 

23 

Datamation 

487 

149 

466 

725.63 

21 

B.  PRESTIGE  OF  UNIVERSITY  AND  JOURNAL  WEIGHTING 

We  examined  the  relationship  between  university  ranking 
and  journal  weighting.  The  variable,  university  ranking, 
has  two  possible  categories  ('’ranked"  or  "not  ranked")  . 
Table  9  shows  the  list  of  IS  journals  in  order  of  the 
"ranked"  column.  Each  column  displays  the  order  placement 
number  and  mean  weight  (in  parenthesis)  for  each  journal. 
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TABLE  9 


PRE8TIGE  OF  UNIVERSITY 


Journal 

Ranked 

Not  Ranked 

Management  Science 

1  (14.01) 

1  (11.62) 

Information  Systems  Research 

2  (12.78) 

5(10.55) 

Communications  of  the  ACM 

3(11.51) 

3(10.79) 

IEEE  Transactions 

4(11.27) 

4(10.58) 

Administrative  Science  Quarterly 

4(11.27) 

6(10.15) 

ACM  Transactions 

5  (10.80) 

9  (9.61) 

ACM  Computing  Surveys 

6(10.75) 

10  (9.45) 

ORSA  Journal  on  Computing 

7(10.69) 

16  (8.24) 

Harvard  Business  Review 

8(10.31) 

8  (9.64) 

Management  Information  Systems  Quarterly 

9(10.96) 

2(10.93) 

Decision  Sciences 

10(10.13 

14  (8.52) 

IEEE  Computer 

11  (9.64) 

12(8.61) 

Sloan  Management  Review 

12  (9.56) 

15(8.46) 

Academy  of  Management  Journal 

13(9.46) 

7(9.71) 

Journal  of  Management  Information  Systems 

14  (9.27) 

11  (8.80) 

Decision  Support  Systems 

15(9.16) 

18  (7.82) 

Accounting  Review 

16  (8.61) 

13  (8.57) 

Omega 

17  (8.08) 

20(7.09) 

Interfaces 

18(7.99) 

17(7.86) 

Information  and  Management 

19  (7.27) 

19  (7.70) 

Journal  of  Systems  Management 

20  (6.53) 

22  (6.58) 

DataBase 

21  (6.17) 

21  (6.95) 

EDP  Analyzer 

22  (5.39) 

23  (6.22) 

Datamation 

23  (4.32) 

24  (5.23) 

By  using  university  rank  as  the  independent  variable,  we 
attempted  to  determine  whether  graduating  from  one  of  the 
prestigiously  ranked  universities  would  have  an  effect  on 
how  respondents  weighted  IS  journals  (Lord,  1993) .  The 
TTEST  was  selected  to  evaluate  this  relationship  since  there 
were  only  two  categories. 

The  null  hypothesis  is  that  there  is  no  difference  in 
the  weighted  means  of  each  IS  journal  and  prestige  of 
university.  The  alternative  hypothesis  is  that  there  is  a 
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statistically  significant  difference  in  the  weighted  means 
of  each  IS  journal  and  prestige  of  university. 


TABLE  10 

PRESTIGE  OF  UNIVERSITY  AND  JOURNAL  WEIGHTING  (TTE8T) 


Journal 

F  Value 

2-Tali  Prob 

Academy  of  Management  Journal 

1.140 

0.616 

Accounting  Review 

1.110 

0.725 

ACM  Computing  Surveys 

1.090 

0.748 

ACM  Transactions 

1.480 

0.120 

Administrative  Science  Quarterly 

1.470 

0.174 

Communications  of  the  ACM 

1.140 

0.559 

DataBase 

1.050 

0.858 

Datamation 

1.470 

0.119 

Decision  Sciences 

1.530 

0.081 

Decision  Support  Systems 

1.540 

0.108 

EDP  Analyzer 

3.280 

*  0.0 

Harvard  Business  Review 

1.190 

0.458 

IEEE  Computer 

2.740 

*0.0 

IEEE  Transactions 

1.470 

0.149 

Information  and  Management 

1.480 

0.146 

Information  Systems  Research 

1.950 

*0.01 

Interfaces 

2.440 

*0.001 

Journal  of  Management  Information  Systems 

1.580 

0.056 

Journal  of  Systems  Management 

2.770 

• 

o 

o 

Management  Information  Systems  Quarterly 

1.920 

*0.005 

Management  Science 

1.840 

*0.0111 

Omega 

2.280 

*0.01 

ORSA  Journal  on  Computing 

1.060 

0.879 

Sloan  Management  Review 

1.250 

0.395 

We  rejected  the  null  hypothesis  for  the  starred  journals 
in  Table  10  with  a  statistical  significance  (alpha)  of  .05 
or  less.  We  failed  to  reject  the  null  hypothesis  for  non- 
starred  journals.  Overall,  in  sixteen  out  of  twenty-four 
journals,  university  ranking  did  not  have  a  statistically 
significant  relationship  with  journal  weightings. 
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C.  REGION  AND  JOURNAL  WEIGHTING 


We  examined  the  relationship  between  region  and  journal 
weighting.  The  variable,  region,  has  four  possible  values 
(East,  Midwest,  South,  and  West) .  Table  11  shows  the  list 
of  IS  journals  in  order  of  the  "East"  column.  All  columns 
display  the  rank  order  placement  number  and  mean  weight  (in 
parenthesis)  for  each  journal. 


TABLE  11 
REGIONS 


Journal 

IEEE  Transactions 

Management  Science 

Communications  of  the  ACM 

Information  Systems  Research 

Administrative  Science  Quarterty 

ACM  Transactions 

ORSA  Journal  on  Computing 

ACM  Computing  Surveys 

Management  Information  Syttems  Quarterly 

Harvard  Business  Review 

Sloan  Management  Review 

Decision  Sciences 

Academy  of  Management  Journal 

IEEE  Computer 

Journal  of  Management  Information  Systems 
Interfaces 

Decision  Support  Systems 
Accounting  Review 
Omega 

Information  and  Management 

Journal  of  Systems  Management 

DataBase 

EDP  Analyzer 

Datamation 


East 

Mdwtst 

South 

West 

1  (13.75) 

7  (10.12) 

6  (9.63) 

8  (10.48) 

2  (13.69) 

1  (13.77) 

1  (11.40) 

2(11.38) 

3(12.11) 

5  (10.50) 

3(10.66) 

3  (11.32) 

4(11.77) 

4(11.12) 

2  (11.24) 

1  (11.71) 

5(10.92) 

3(12.04) 

7(9.27) 

11  (10.24) 

8(10.81 } 

8(10.32) 

12  (6.78) 

5  (10.85) 

7(10.26) 

16  (8.35) 

14  (8.46) 

13  (9.88) 

8(10.09) 

9(9.77) 

10  (9.06) 

4(11.31) 

9(10.03) 

2(12.32) 

5(9.78) 

9  (10.33) 

10  (9.78) 

11  (9.46) 

4(9.84) 

6(10.79) 

11  (9.61) 

15  (8.41) 

16(8.13) 

12(9.96) 

12(9.46) 

12  (9.37) 

13  (8.66) 

15(9.31) 

13  (9.26) 

10  (9.72) 

9  (9.14) 

7  (10.71) 

14  (9.10) 

13  (9.00) 

11  (9.05) 

17(9.00) 

15  (8.71) 

14  (8.92) 

8(9.21) 

16(9.04) 

16  (8.07) 

18(7.27) 

17  (7.69) 

18(8.85) 

17(7.95) 

17  (8.07) 

15  (8.34) 

14(9.50) 

18  (7.58) 

8  (9.96) 

21  (6.80) 

10(10.26) 

19  (7.47) 

18(7.27) 

19  (7.58) 

22  (7.71) 

20(7.23) 

19  (7.13) 

18  (7.60) 

19  (8.54) 

21  (6.34) 

21  (5.74) 

22  (6.79) 

23(7.43) 

22(6.18) 

20  (6.47) 

20(7.24) 

21  (7.76) 

23  (4.52) 

22  (4.92) 

23  (6.37) 

20  (8.27) 

24  (3.00) 

23  (4.64) 

24  (5.43) 

24(6.47) 
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Analysis  of  variance  (ANOVA)  was  selected  as  the 
appropriate  statistical  test  because  the  independent 
variable  academic  region  had  four  possible  categories, 
null  hypothesis  is  that  there  is  no  difference  in  the 
weighted  means  of  each  IS  journal  by  region.  The 
alternative  hypothesis  is  that  there  is  a  statistically 
significant  difference  in  the  weighted  means  of  each  IS 
journal  and  region. 


TABLE  12 

REGION  AND  JOURNAL  WEIGHTING  (ANOVA) 


Journal 

F  Value 

Slg.  of  F 

Academy  of  Management  Journal 

0.403 

0.751 

Accounting  Review 

4.657 

•0.007 

ACM  Computing  Surveys 

2.168 

0.099 

ACM  Transactions 

1.260 

0.296 

Administrative  Science  Quarterly 

1.649 

0.187 

Communications  of  the  ACM 

0.222 

0.881 

Data  Base 

1.716 

0.172 

Datamation 

3.818 

*0.013 

Decision  Sciences 

1.383 

0.255 

Decision  Support  Systems 

2.175 

0.100 

EDP  Analyzer 

3.870 

•0.014 

Harvard  Business  Review 

1.266 

0.292 

IEEE  Computer 

0.863 

0.465 

IEEE  Transactions 

1.206 

0.315 

Information  and  Management 

2.299 

0.084 

Information  Systems  Research 

0.060 

0.981 

Interfaces 

1.458 

0.233 

Journal  of  Management  Information  Systems 

0.332 

0.802 

Journal  of  Systems  Management 

3.628 

•0.017 

Management  Information  Systems  Quarterly 

1.277 

0.288 

Management  Science 

1.760 

0.161 

Omega 

0.098 

0.961 

ORSA  Journal  on  Computing 

0.934 

0.433 

Sloan  Management  Review 

2.186 

0.097 

The 
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We  rejected  the  null  hypothesis  for  the  starred  journals 


in  Table  12  with  a  statistical  significance  (alpha)  of  .05 
or  less.  We  failed  to  reject  the  null  hypothesis  for  non- 
starred  journals.  Overall,  region  did  not  have  a 
statistically  significant  relationship  to  journal  weighting. 


D.  ACADEMIC  SPECIALTY  AMD  JOURNAL  WEIGHTING 

We  examined  the  relationship  between  academic  specialty 
and  journal  weighting.  The  variable,  academic  specialty, 
fell  into  six  nominal  categories  (8.1,  8.2,  8.3,  8.4,  8.5 
and  unknown) .  Table  13  shows  the  list  of  IS  journals  in 
order  of  the  "8.1"  column.  All  columns  display  the  rank 
order  placement  number  and  mean  weight  (in  parenthesis)  for 
each  journal. 
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TABLE  13 


ACADEKIC  SPECIALTY 


Journal 

•.i 

U 

U 

u 

84 

Unknown 

Managamant  Science 

1  (12.86) 

1  (11.66) 

1  (12.70) 

3(8.67) 

2(29.00) 

IEEE  Transaction* 

2(12.16) 

11  (8.91) 

4(10.50) 

6(740) 

3(24.50) 

Management  Information  Systems  Quarterly 

3(11.73) 

6  (10.16) 

8(941) 

1  (10.00) 

2(6.00) 

13(1147) 

CommunicatfonaaftieACM 

4(11.52) 

3(11.09) 

3  (10.73) 

7(7.00) 

1(7.00) 

8  (15.00) 

MIINHunVI  oGWnOI  \JumWTTj 

8(1142) 

4(10.74) 

10(9.40) 

5(8.00) 

7(16.00) 

Information  Systems  Raaaareh 

6(10.96) 

2(11.27) 

2(12.18) 

11  (5.50) 

7  (16.00) 

ACM  Transactors 

7  (10.87) 

12(8.76) 

5  (10.20) 

6(7.50) 

6  (17.50) 

ORSA  Journal  on  Computing 

8(10.86) 

19  (7.09) 

14(9.04) 

7(7.00) 

1  (30.00) 

ACM  Computing  Surveys 

8(10.47) 

8(9.34) 

6(948) 

2(9.67) 

3(5.00)  10(1343) 

Aoadamy  of  Managamant  Journal 

10(9.82) 

5(10.41) 

16  (8.69) 

4(8.33) 

19(5.33) 

Harvard  Businas*  Review 

11  (9.73) 

7(9.84) 

7(946) 

1  (10.00) 

9  (14.67) 

Decision  Sdenoes 

12(9.62) 

15(8.09) 

11  (9.39) 

8(6.87) 

7(16.00) 

IEEE  Computar 

13  (8.16) 

14  (8.51) 

15(8.95) 

9  (8.50) 

4(4.00) 

4(23.00) 

Journal  of  Managamant  information  Systems 

14  (8.96) 

10(8.96) 

12(9.12) 

10(5.67) 

14(11.00) 

Decision  Support  Systams 

IS  (8.65) 

16  (7.32) 

13(9.07) 

12(5.00) 

12  (12.67) 

Sloan  Managamant  Review 

16  (8.37) 

13  (8.64) 

9(9.86) 

4(8.33) 

11  (13.00) 

Accounting  Review 

17  (8-10) 

9(9.04) 

18(846) 

3(8.67) 

12  (12.67) 

Interfaces 

18(7.83) 

16  (7.73) 

17(849) 

14(4.50) 

15  (10.00) 

Omega 

19  (7.52) 

17(7.39) 

21  (7.27) 

15  (4.00) 

5(20.00) 

Information  and  Management 

20(7.43) 

16(7.73) 

19  (7.58) 

16(3.00) 

17(9.00) 

DataBase 

21  (7.30) 

20  (6.21) 

20(7.36) 

16(3.00) 

1(7.00) 

18(8.00) 

Journal  of  Systams  Management 

22(6.81) 

21  (6.13) 

23  (6.58) 

10(5.67) 

16(9.33) 

EDP  Analyzer 

23(4.97) 

22(5.92) 

22(6.83) 

1  (10.00) 

2(600) 

21(3.00) 

Datamation 

24(4.55) 

23(4.67) 

24  (5.67) 

13(4.67) 

2(6.00) 

20(4.00) 

Analysis  of  variance 

(ANOVA) 

was 

selected  as 

the 

appropriate  statistical  test  because  the  independent 


variable  academic  specialty  had  six  possible  categories. 

The  null  hypothesis  is  that  there  is  no  difference  in  the 
weighted  means  of  each  IS  journal  and  academic  specialty. 
The  alternative  hypothesis  is  that  there  is  a  statistically 
significant  difference  in  the  weighted  means  of  each  IS 


journal  according  to  academic  specialty. 
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TABLE  14 


ACADEMIC  SPECIALTY  AMD  JOURNAL  WEIGHTING  (ANOVA) 


Journal 

F  Value 

Slg.  of  F 

Academy  of  Management  Journal 

1.485 

0.234 

Accounting  Review 

0.660 

0.522 

ACM  Computing  Surveys 

0.194 

0.90 

ACM  Transactions 

1.065 

0.351 

Administrative  Science  Quarterly 

1.101 

0.339 

Communications  of  the  ACM 

0.908 

0.441 

DataBase 

1.721 

0.170 

Datamation 

0.422 

0.738 

Decision  Sciences 

0.516 

0.599 

Decision  Support  Systems 

2.280 

0.111 

EDP  Analyzer 

0.239 

0.869 

Harvard  Business  Review 

0.006 

0.994 

IEEE  Computer 

1.168 

0.329 

IEEE  Transactions 

1.580 

0.214 

Information  and  Management 

3.228 

*0.045 

Information  Systems  Research 

3.077 

3.077 

Interfaces 

1.185 

0.312 

Journal  of  Management  Information  Systems 

1.304 

0.277 

Journal  of  Systems  Management 

0.237 

0.790 

Management  Information  Systems  Quarterly 

4.389 

*0.006 

Management  Science 

0.455 

0.636 

Omega 

1.619 

0.209 

ORSA  Journal  on  Computing 

0.739 

0.484 

Sloan  Management  Review 

0.582 

0.561 

We  rejected  the  null  hypothesis  for  the  starred  journals 
in  Table  14  with  a  statistical  significance  (alpha)  of  .05 
or  less.  We  failed  to  reject  the  null  hypothesis  for  non- 
starred  journals.  Overall,  academic  specialty  did  not  have 
a  statistically  significant  relationship  in  journal 
weighting. 


E.  DEGREE  TYPE  AND  JOURNAL  WEIGHTING 

We  examined  the  relationship  between  degree  type  and 
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journal  weighting.  The  variable,  degree  type  has  five 
possible  values  (Information  Systems,  Business,  Social 
Sciences,  Mathematic  and  Physical  Sciences  and  Unknown) . 
Table  15  shows  the  list  of  IS  journals  in  order  of  the  MIS" 
column.  All  columns  display  the  rank  order  placement  number 
and  mean  weight  (in  parenthesis)  for  each  journal. 


TABLE  15 


DEGREE  TYPE 
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Analysis  of  variance  (ANOVA)  was  selected  as  the 
appropriate  statistical  test  because  the  independent 
variable  degree  type  had  five  possible  categories.  The  null 
hypothesis  is  that  there  is  no  difference  in  the  weighted 
means  of  each  IS  journal  and  degree  type.  The  alternative 
hypothesis  is  that  there  is  a  statistically  significant 
difference  in  the  weighted  means  of  each  IS  journal  and 
degree  type. 


TABLE  16 

DEGREE  TYPE  AMD  JOURNAL  WEIGHTING  (ANOVA) 


Journal 

F  Value 

Slg.  of  F 

Academy  of  Management  Journal 

2.305 

0.086 

Accounting  Review 

2.020 

0.126 

ACM  Computing  Surveys 

0.324 

0.808 

ACM  Transactions 

0.009 

0.999 

Administrative  Science  Quarterly 

0.668 

0.575 

Communications  of  the  ACM 

0.955 

0.418  1 

Data  Base 

2.010 

0.120 

Datamation 

1.449 

0.235 

Decision  Sciences 

0.926 

0.433 

Decision  Support  Systems 

1.833 

0.150 

EDP  Analyzer 

1.864 

0.146 

Harvard  Business  Review 

0.696 

0.557 

IEEE  Computer 

1.115 

0.349 

IEEE  Transactions 

0.306 

0.821 

Information  and  Management 

2.942 

•0.039 

Information  Systems  Research 

0.692 

0.560 

Interfaces 

1.250 

0.298 

Journal  of  Management  Information  Systems 

0.095 

0.962 

Journal  of  Systems  Management 

3.159 

•0.03 

Management  Information  Systems  Quarterly 

2.499 

0.065 

Management  Science 

1.042 

0.379 

Omega 

0.159 

0.923 

ORSA  Journal  on  Computing 

0.929 

0.436 

Sloan  Management  Review 

3.597 

*  0.017 
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We  rejected  the  null  hypothesis  for  the  starred  journals 
in  Table  16  with  a  statistical  significance  (alpha)  of  .05 
or  less.  We  failed  to  reject  the  null  hypothesis  for  non- 
starred  journals.  Overall,  degree  type  did  not  have  a 
statistically  significant  relationship  in  journal  weighting. 

F.  DEGREE  DATE  AND  JOURNAL  WEIGHTING 

We  examined  the  relationship  between  degree  date  and 
journal  weighting.  The  variable,  degree  date  has  five 
possible  values  (1950's,  1960's,  1970's,  1980's,  and 
1990's).  Table  17  shows  the  list  of  IS  journals  in  order  of 
the  "1950's"  column.  All  columns  display  the  rank  order 
placement  number  and  mean  weight  (in  parenthesis)  for  each 
journal. 
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TABLE  17 


DEGREE  DATE 


Journal 

ISM 

1880 

1870 

1888 

1888 

Information  Syvtama  Raaaarch 

1  (1243) 

2(12.53) 

3(11.08 ) 

3(1048) 

3(1241) 

MtMQamnt  Sdmoc 

2(11.83) 

1  (1244) 

1  (12.51) 

1(1248) 

103.78) 

ACM  Compu8ng  Survay* 

3(10.13) 

12(8.70) 

6(1043) 

10(8.46) 

6(10.73) 

Managamant  Information  Syatama  Quartarty 

4(1040) 

10(1041) 

8(1044) 

7(8.81) 

110040) 

ACM  TranaacOona 

S  (9.88) 

7(10.80) 

11  (8.63) 

6  (10.04) 

7(11.13) 

IEEE  Transaction* 

8(843) 

3(1240) 

2(1143) 

6(10.03) 

401-88) 

Communlcaaora  of  8w  ACM 

7(8.78) 

8(1140) 

6(1048) 

2(10.88) 

20243) 

Harvard  Bualnaa*  RavWw 

8(843) 

8(10.66) 

10  (844) 

8(8.81) 

601-07) 

Journal  of  Man^jamant  Information  fiyilrm 

8(7.78) 

16(848) 

13(848) 

18(848) 

12(844) 

Acadamy  at  Managamant  Journal 

10  (7.71) 

18(741) 

7(10.18) 

8(8.57) 

6(1143) 

Omaga 

11(7.80) 

17(746) 

21(743) 

20(748) 

18(745) 

IntiKteoot 

12(740) 

22(5.63) 

17(8.30) 

18(843) 

17(8.75) 

IEEE  Computar 

13(7.00) 

6  (10.74) 

18(8.68) 

14(8.80) 

16(8.44) 

AdmlnlstraBva  Sdanoa  Quartarty 

14(8.71) 

4(11.50) 

4(10.80) 

4  0042) 

5(1144) 

Sloan  Managamant  fMaw 

16(8.43) 

13(844) 

14(8.06) 

16(8.56) 

10(10.07) 

Dor  it  km  Sdonooi 

18(548) 

11  (841) 

12(843) 

11(848) 

18(841) 

Information  and  Managanwnt 

17(5.71) 

18(6.88) 

18(6.15) 

18(7.47) 

18(7.73) 

Journal  of  Syswms  Managamant 

18(540) 

20(8.47) 

22(748) 

22(848) 

21  (8-00) 

Data  Basa 

18(548) 

21  (8.00) 

18  (848) 

21  (6  33) 

20(7.12) 

ORSA  Journal  on  Computing 

20(500) 

14(8.46) 

8(8.86) 

12  (8.87) 

14(8.50) 

Accounting  Ravtaw 

21(4.00) 

8  (10.53) 

20(7.71) 

13(8.82) 

15(844) 

EOP  Analyzar 

22(445) 

24(440) 

23(7.08) 

23(6.86) 

22(540) 

Dadsion  Support  Syatama 

23(4.14) 

16(848) 

15(8.78) 

17(840) 

13(843) 

Datamation 

24(3.44) 

23(4.62) 

24(5.33) 

24(4.61) 

23(444) 

Analysis  of  variance  (ANOVA)  was  selected  as  the 
appropriate  statistical  test  because  the  independent 
variable  degree  date  had  five  possible  categories.  The  null 
hypothesis  is  that  there  is  no  difference  in  the  weighted 
means  of  each  IS  journal  and  degree  date.  The  alternative 
hypothesis  is  that  there  is  a  statistically  significant 
difference  in  the  weighted  means  of  each  IS  journal  and 
degree  date. 
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TABLE  18 


DEGREE  DATE  AMD  JOURNAL  WEIGHTING  (AMOVA) 


Journal 

F  Value 

«g.of  F 

Acadamy  of  Managamant  Journal 

1.888 

0.156 

Accounting  Review 

1.582 

0.184 

ACM  Computing  Survey* 

0.317 

0.866 

ACM  Transactions 

0.324 

0.661 

Administrative  Solanos  Quarterly 

1.011 

0.406 

Communications  oMha  ACM 

1.259 

0.288 

Data  Bass 

2.474 

*0.048 

Datamation 

0.335 

0.884 

Decision  Sdancss 

0.858 

0.434 

Dsdsion  Support  Systsms 

1.835 

0.127 

EDP  Analyzer 

0.886 

0.487 

Harvard  Business  Review 

0.513 

0.726 

IEEE  Computer 

1.320 

0.267 

IEEE  Transactions 

0.528 

0.714 

Information  end  Managemsnt 

1.242 

0.297 

Information  Systsms  Research 

0.683 

0.598 

Interfaces 

1.570 

0.187 

Journal  of  Management  Information  Systsms 

1.332 

0.261 

Journal  of  Systems  Managemsnt 

0.803 

0.464 

Management  Information  Systsms  Quarterly 

0.341 

0.850 

Management  Science 

0.190 

0.943 

Omega 

0.061 

0.893 

ORSA  Journal  on  Computing 

0.443 

0.777 

Sloan  Management  Review 

1.046 

0.386 

We  rejected  the  null  hypothesis  for  the  starred  journals 
in  Table  18  with  a  statistical  significance  (alpha)  of  .05 
or  less.  We  failed  to  reject  the  null  hypothesis  for  non- 
starred  journals.  Overall,  degree  date  did  not  have  a 
statistically  significant  relationship  in  journal  weighting. 

In  summary,  none  of  the  independent  variables  (prestige 
of  university,  region,  academic  specialty,  degree  type,  or 
degree  date)  had  an  overall  statistically  significant 
relationship  with  the  journal  weightings. 


V.  DISCUSSION  AND  ANALYSIS 


A.  COMPARISON  OP  RANKINGS 

Table  19  gives  the  results  of  six  studies  in  which  IS 
journals  were  ranked.  These  studies  were  conducted  from 
1983  to  1993.  There  is  a  noticeable  similarity  in  the 
rankings.  All  of  the  rankings  have  at  least  five  of  the  top 
ten  journals  in  common,  although  not  in  the  same  order. 
Management  Science,  MISQ,  and  Communications  of  the  ACM  were 
among  the  top  ten  journals  in  all  six  studies.  Harvard 
Business  Review,  Computing  Surveys  and  Data  Base  were 
present  in  five  and  four,  respectively,  of  the  six  rankings. 
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TABLE  19 


THE  TOP  TEH  JOURNALS  AS  COMPARED  IN  SIX  STUDIES 


Doke  & 
Luke 

Vogel  & 
Wetherbe 

Hamilton 
Ives  & 

Shim  et 
al. 

Ramesh  & 
Stohr 

Hayes  & 
Huskey 

MISQ 

Comms  of 
the  ACM 

MISQ 

Mgmt 

Science 

Mgmt 

Science 

Mgmt 

Science 

J  of  MIS 

Harvard 

Bus  Rev 

EDP 

Analyzer 

Comms  of 
the  ACM 

Ops 

Research 

IS 

Research 

Comms  of 
the  ACM 

MISQ 

Data 

Base 

MISQ 

Comms  of 
the  ACM 

Comms  of 
the  ACM 

Decision 

Sciences 

Info  & 

Mgmt 

Mgmt 

Science 

Harvard 

Bus  Rev 

J  of  the 
ACM 

IEEE 

Trans 

Mgmt 

Science 

Sloan 

Mgmt  Rev 

Data¬ 

mation 

Decision 

Sciences 

ACM  Trans 
Database 

Admin 

Science 

Qtrly 

Data¬ 

mation 

Mgmt 

Science 

Harvard 

Bus  Rev 

Sloan 

Mgmt  Rev 

IEEE 

Trans 

S/W  Eng 

ACM 

Trans 

Harvard 
Bus  Rev 

J  of  Data 
Education 

Computing 

Surveys 

Art  Intel 

MISQ 

J  of 

Comp  IS 

Data¬ 

mation 

Comms  of 
the  ACM 

Computing 

Surveys 

Cognitive 

Science 

ACM  Comp 
Surveys 

J  of  IS 
Mgmt 

Data  Base 

Trans  on 
Dbase  Sys 

IBM  Sys 
Journal 

MISQ 

Harvard 
Bus  Rev 

Data 

Base 

Trans  on 
Dbase  Sys 

Info  and 
Mgmt 

Data  Base 

Computing 

Surveys 

Acad  of 
Mgmt  J 
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B.  INTRODUCTION  OF  NEW  JOURNALS 


Some  of  the  variations  in  the  top  ten  rankings  may  be 
attributed  to  the  introduction  of  new  journals.  Not  only 
are  new  journals  created  but  some  journals  cease  to  be 
published.  Exact  beginning  or  ending  publication  dates  of 
the  journals  listed  in  all  six  surveys  were  not  researched, 
but  at  least  one  new  journal.  Information  Systems  Research 
was  included  in  the  last  two  studies  (Ramesh  and  Stohr, 

1987,  and  Hayes  and  Huskey,  1993) .  Journals  need  time  to 
build  a  reputation,  Information  Systems  Research  may  not 
have  been  established  enough  to  make  the  top  ten  journal 
list  in  Ramesh  and  Stohr' s  study,  but  it  did  in  our  study. 
This  explains  why  earlier  studies  would  have  similar  core 
lists  of  journals,  they  may  have  used  a  smaller  list  due  to 
the  lack  of  availability  of  newly  published  IS  journals.  It 
was  also  noted  that  EDP  Analyzer  had  changed  its  name  to  ADP 
Analyzer . 


C.  SAMPLE  DIFFERENCES 

In  addition  to  the  introduction  of  new  journals, 
differences  in  the  samples  taken  by  each  study  could  account 
for  the  variation  in  ranking.  Doke  and  Luke  (1987)  surveyed 
deans  of  business  schools  (with  instructions  to  distribute 
them  to  professors,  associate  professors  and  assistant 
professors) .  How  the  deans  chose  to  distribute  the  surveys 
among  the  faculty  is  unknown.  Doke  and  Luke  (1987)  sent 
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their  surveys  primarily  to  business  schools,  whereas  the 
surveys  in  our  study  went  directly  to  MIS  faculty  . 

Hamilton  and  Ives  (1983)  sent  surveys  to  "knowledgeable 
recognized  experts."  Shim  et  al.  (1987)  used  only 
"outstanding"  senior  researchers  for  their  survey.  They 
excluded  any  junior  faculty  or  lesser  known  senior  faculty 
opinions,  even  though  the  latter  may  be  the  decision  makers 
in  hiring,  tenure  or  promotion  situations. 

The  survey  in  our  study  validates  the  findings  of 
previous  studies  of  IS  journal  rankings.  MIS  academicians 
have  not  significantly  shifted  their  subjective  prestige 
ranking  of  IS  journals  in  the  last  decade.  The  results  show 
that  half  of  the  journals  thought  of  as  prestigious  in  five 
other  surveys  continue  to  be  thought  of  highly  in  our 
survey . 

D.  VALUE  OP  SURVEY  METHODOLOGY 

The  main  value  of  this  study  is  the  use  of  prestige 
weights  rather  than  ranking.  Prestige  weights  allow 
decision  makers  to  quantify  the  differential  prestige  of 
journals  in  hiring,  promotion  or  tenure  decisions.  If  a 
candidate  for  promotion  has  articles  in  different  journals, 
the  mu  frequencies  of  the  articles  in  journals  can  be 
multiplied  by  the  prestige  weight  to  obtain  a  more 
meaningful  indicator  of  the  value  of  a  publishing  record.  A 
ranking  of  journals  would  not  have  given  any  distance  type 
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measure  between  the  journals,  only  the  order  in  which  they 
are  perceived.  A  weighting  gives  both  a  distance  between 
journals  in  addition  to  order  of  preference. 

The  weighting  of  IS  journals  can  also  be  valuable  to 
those  who  are  evaluating  prestige  and  productivity  of  IS 
departments.  (Allen,  1993) 

E.  SURVEY  DIFFERENCES 

Several  rankings  of  IS  journals  have  been  conducted 
(Hamilton  and  Ives,  1983;  Doke  and  Luke,  1987;  Vogel  and 
Wetherbe,  1984) .  Our  study  used  a  different  survey 
methodology.  We  used  a  weighting  to  develop  a  ranking.  As 
discussed  previously,  the  weighting  can  provide  a  tool  for 
decision  makers  to  use. 

We  used  a  random  sample  of  MIS  professionals  from  the 
1992  McGraw-Hill  Directory  of  Management  Information  Systems 
Faculty.  Ramesh  and  Stohr  (1987)  also  used  a  random  sample 
from  the  MIS  directory,  but  limited  their  sample  to  CS 
professionals.  We  used  a  larger  sample  size  (400)  compared 
to  the  200  used  by  Ramesh  and  Stohr  (1987) .  The  sample  of 
400  in  this  study  was  not  limited  to  one  segment  of  the  MIS 
community,  but  included  faculty  members  from  all  academic 
ranks . 

Our  study  differed  from  earlier  studies  in  that  it 
compared  respondents  to  non- respondents  in  regard  to 
prestige  of  university,  region,  academic  specialty,  degree 
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type,  and  degree  date.  We  determined  if  any  of  the 
independent  variables  were  statistically  significant  between 
respondents  anl  non- respondents .  The  database  allowed  us  to 
compare  weights  calculated  by  cross-referencing  respondents 
with  certain  variables.  As  far  as  we  know,  this  has  not 
been  done  in  any  previous  studies.  As  seen  by  our  results 
in  Tables  8  through  18,  there  was  no  significant  difference 
in  the  ranking  derived  from  the  mean  weights  between  the 
different  categories  of  each  variable. 

The  comparison  of  independent  variables  showed  no 
significant  differences  in  IS  journal  weightings.  The 
respondent's  prestige  of  university,  region,  academic 
specialty,  degree  type,  or  degree  date  had  no  significant 
impact  on  how  the  journals  were  weighted.  This  proves  that 
our  random  sample  represented  an  even  population 
distribution  between  all  the  categories  of  the  independent 
variables . 

F .  LESSONS  LEARNED 

There  are  several  lessons  that  were  learned  during  the 
survey  process.  But  there  were  three  aspects  of  our  study, 
had  we  the  opportunity,  would  do  differently. 

One  of  the  studies  (Hawkins  et  al.,  1973)  of  economic 
journals  used  an  interesting  tactic.  The  inclusion  of 
fictitious  journals  in  their  core  list  of  journals  proved 
that  respondents  do  recognize  general  academic  journals  more 
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than  specialized  academic  journals.  It  was,  however, 
surprising  that  one  of  the  fictitious  journals  (J.E.S.T.) 
was  recognized  and  ranked  in  the  top  third  of  their  list  of 
economic  journals. 

Although  we  accepted  suggestions  for  additional  journals 
to  the  core  list  of  IS  journals  from  respondents  in  the  test 
survey,  we  did  not  leave  an  open-ended  opportunity  for 
respondents  in  the  actual  survey.  A  more  complete  and 
diverse  list  of  IS  journals  could  have  been  compiled  and  we 
would  have  avoided  any  bias  created  by  limiting  the  list  of 
core  journals. 

If  we  had  started  the  survey  process  earlier,  a  follow 
up  survey  could  have  been  done.  Even  a  reminder  (with  the 
original  survey)  could  have  been  sent  to  those  who  had  not 
responded  after  a  set  period  of  time  and  increased  our 
response  rate. 

G.  CONCLUSION 

The  weighted  ranking  of  IS  journals  could  be  used  by  MIS 
faculty  when  making  decisions  in  regards  to  hiring, 
promotion,  and  tenure.  This  was  accomplished  by  using  a 
survey  asking  respondents  to  weight  journals  according  to 
their  perceived  prestige  value.  A  ranking  was  formed  based 
on  the  mean  weights  of  the  IS  journals.  Statistical 
analysis  showed  that  independent  variables  did  not  have  a 
significant  impact  on  the  perceived  prestige  of  journals. 
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Professionals  can  use  this  study  for  exercises  in 
performance  evaluation  of  research  organizations.  We  have 
researched  several  methodologies  in  other  disciplines  for 
ranking  journals.  Our  research  methodology  uses  aspects  of 
these  different  methodologies  and  can  provide  a  tool  for 
measuring  performance  evaluation. 

Currently  DoD  is  trying  to  use  the  CIM  initiative  and 
business  re-engineering  to  improve  business  practices. 

These  two  processes  are  intertwined  to  evaluate  performance 
and  make  it  more  efficient.  Our  study  is  of  particular 
interest  to  performance  evaluations  in  the  context  of  DoD's 
policy  of  re-engineering  business  processes.  The  research 
methodology  explored  techniques  to  quantify  an 
impressionistic  rating.  This  would  be  useful  in  quantifying 
ratings  of  performance  evaluations  in  management  of  DoD 
laboratories  and  schools. 
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APPENDIX  A 


Dear  Colleague: 

We  are  sending  out  this  survey  as  part  of  a  thesis  project  that 
will  attempt  to  rank  the  Information  System  core  journals  in  order 
to  show  which  publications  are  held  in  the  highest  esteem  by 
Management  Information  System  faculty  members.  This  should  be  of 
great  value  to  researchers  looking  to  publish  in  order  to  get 
tenure . 

The  following  list  of  Information  Systems  publications  is 
provided  for  you  to  rank  (one  through  ten)  in  accordance  with  your 
judgement  of  the  importance  of  their  contributions  to  the  field. 
Of  course,  the  importance  of  the  individual  publications  in  certain 
categories  may  vary  greatly,  but  think  in  terms  of  their  overall 
importance . 

Please  put  the  rank  number  in  the  space  provided  after  the 
publication  title. 

After  completing  the  form,  please  return  it  via  the  self- 
addressed  stamped  envelope.  Please  indicate  if  you  would  like  a 
copy  of  the  results. 

We  sincerely  thank  you  for  all  your  assistance  in  this  project. 
If  you  have  any  questions  please  call  us  at  (408)  656-7124. 
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Name _ 

JOURNAL 

Academy  of  Management  Journal 

Accounting  Review 

ACM  Computing  Surveys 

Administrative  Science  Quarterly 

Communications  of  the  ACM 

Data  Base 

Datamation 

Decision  Sciences 

EDP  Analyzer 

Harvard  Business  Review 

IEEE  Computer 

Information  and  Management 

Information  Systems  Research 

Interfaces 

Journal  of  Management  Information  Systems 
Journal  of  Systems  Management 
Management  Information  Systems  Quarterly 
Management  Science 
Omega 

Sloan  Management  Review 


RANKING  (top  ten  only) 
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APPENDIX  B 


V 


Dear  Colleague: 

We  are  sending  out  this  survey  as  part  of  a  thesis  project  that 
will  attempt  to  rank  the  Information  System  core  journals  in  order 
to  show  which  publications  are  held  in  the  highest  esteem  by 
Management  Information  System  faculty  members.  This  should  be  of 
great  value  to  researchers  looking  to  publish  in  order  to  get 
tenure . 

On  the  form  please  assign  weights  to  the  following  kinds  of 
information  system  publications  according  to  your  judgment  of  their 
prestige  in  the  field.  Of  course,  the  importance  of  individual 
publications  in  each  category  varies  greatly,  but  please  think  in 
terms  of  their  average  importance.  Use  the  Management  Information 
Systems  Quarterly  (MISQ)  as  your  standard  of  reference.  A  weight 
of  10  has  been  arbitrarily  assigned  to  MISQ,  so  that  a  publication 
only  half  as  important  as  MISQ  should  be  assigned  a  weight  of  5,  a 
publication  twice  as  important  as  MISQ  should  be  assigned  a  weight 
of  20,  and  so  forth.  Negative  values  are  acceptable. 

If  you  do  not  know  enough  about  a  journal  to  assign  a  weight  to 
its  articles,  please  place  an  X  in  the  space  provided  for  the 
weight . 

Remember,  a  weight  of  10  has  arbitrarily  been  assigned  to 
articles  in  the  MISQ.  Use  this  as  your  standard  for  assigning 
other  weights. 

After  completing  the  form,  please  return  it  via  the  self- 
addressed  stamped  envelope.  Please  indicate  if  you  would  like  a 
copy  of  the  results. 

We  sincerely  thank  you  'or  all  your  assistance  in  this  project. 
If  you  have  any  questions  please  call  us  at  (408)  656-7124. 
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JOURNAL 


WEIGHT 


Academy  of  Management  Journal 

Accounting  Review 

ACM  Computing  Surveys  * 

Administrative  Science  Quarterly 

Communications  of  the  ACM 

Data  Base 

Datamation 

Decision  Sciences 

EDP  Analyzer  ** 

Harvard  Business  Review 
IEEE  Computer  *** 

Information  and  Management 
Information  Systems  Research 
Interfaces 

Journal  of  Management  Information  Systems 
Journal  of  Systems  Management 
Management  Information  Systems  Quarterly 
Management  Science 
Omega 

Sloan  Management  Review 


10 


*  ACM:  Association  for  Computing  Machinery 

**  EDP:  Electronic  Data  Processing 

***  IEEE:  Institute  of  Electronic  and  Electrical  Engineering 


APPENDIX  C 


DEPARTMENT  OF  THE  NAVY 

NAVAL  AOSTGNAOUATV  SCHOOL 
MONTE  SKY.  CA  S3S4>9100 


Dear  Colleague, 

I  beg  your  indulgence  of  an  imposition  upon  your  time  and  patience.  I  am  asking  you  to 
assign  relative  weights  to  a  selection  of  IS  journals. 

You  may  find  this  exercise  to  be  a  bit  artificial.  I  cannot  dismiss  that  assessment. 
Nonetheless,  I  speculate  that  many  of  us  form  a  ranking  or  rating  of  IS  publications  in  our 
minds,  however  impressionistic  they  may  be.  We  may  do  it  when  evaluating  a  colleague's 
vita,  when  assessing  the  credibility  of  an  IS  department  or  when  musing  about  your  own 
value  in  the  academic  market. 

This  is  merely  an  attempt  to  impose  a  quantification  exercise  on  what  we  may  already  be 
doing.  This  rating  technique  has  been  borrowed  from  reference  disciplines  in  the  social 
sciences.  I  just  want  to  see  if  it  is  applicable  within  the  IS  community. 

On  the  attached  sheet,  please  assign  a  weight  to  each  journal  about  which  you  have  an 
opinion.  If  you  have  no  opinion  about  a  journal,  put  an  X  in  the  space.  Use  the 
Management  Information  Systems  Quarterly  as  your  standard  of  reference.  A  weight  of  10 
has  been  arbitrarily  assigned  to  M1SQ.  A  publication  that  you  regard  as  half  the  worth  of 
MISQ  would  get  a  5.  A  publication  you  deem  to  be  twice  the  value  of  MISQ  would  get  a  20. 
And  so  on.  Negative  values  are  acceptable 

You  may  adopt  any  perspective  or  criteria  you  wish  as  the  yardstick  by  which  you  are 
assessing  these  journals,  e.g.  value  to  the  IS  community,  impact  of  published  articles,  quality 
of  research,  standing  of  authors,  etc.  The  list  of  IS  journals  on  the  accompanying  score  sheet 
has  been  adopted  from  Davis(1980),  Hamilton  &  Ives(1983)  and  Vogel  &  Wetherbe(1984). 


Thank  You. 

William  James  Haga 

Associate  Professor,  Information  Systems 


WEIGHT 


JOURNAL 

Academy  of  Management  Journal 

Accounting  Review 

ACM  Computing  Surveys 

ACM  Transactions  (various  specialties) 

Administrative  Science  Quarterly 

Communications  of  the  ACM 

Data  Base 

Datamation 

Decision  Sciences 

Decision  Support  Systems 

EDP  Analyzer 

Harvard  Business  Review 

JFJRE  Computer 

tfff  Transactions  (various  specialties) 
Information  and  Management 
Information  Systems  Research 
Interfaces 

Journal  of  Management  Information  Systems 
Journal  of  Systems  Management 
Management  Information  Systems  Quarterly 
Management  Science 
Omega 

ORSA  Journal  on  Computing 
Sloan  Management  Review 
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APPENDIX  D 


The  following  describes  each  academic  specialty: 

Academic  Specialty  8.1:  Design  and  Evaluation  of 
Information  Systems,  (The  development  of  approaches  for  the 
analysis,  design,  specification,  and  evaluation  of  computer- 
assisted  information  systems) . 

Academic  Specialty  8.2:  The  Interaction  of  Information 
Systems,  (The  investigation  of  the  relationships  and 
interactions  among  four  major  components:  information 
systems,  information  technology,  organizations  and  society. 
The  focus  is  on  the  interrelationships,  not  on  the 
components  themselves) . 

Academic  Specialty  8.3:  Decision  Support  Systems,  (The 
development  of  approaches  for  applying  information  systems 
technology  to  increase  the  effectiveness  of  decision-makers 
in  situations  where  the  computer  system  can  support  and 
enhance  human  judgements  in  the  performance  of  tasks  that 
have  elements  which  cannot  be  specified  in  advance) . 

Academic  Specialty  8.4:  Office  Information  Systems,  (The 
study  and  development  of  information  systems  for  office 
work.  Such  systems  are  concerned  with  the  support  of,  and 
communication  in  connection  with,  human  activities  in  an 
organization.  They  are  characterized  by,  among  other 
things,  variety,  informality  and  irregularity,  but  often 
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interact  strongly  with  the  more  orderly,  formal  and 
predictable  computer-based  information  systems  used  in  that 
organization) . 

Academic  Specialty  8.5:  Information  Systems  in  Public 
Administration,  (Information  systems  in  public 
administration  at  international,  national,  regional,  and 
local  levels.  The  relationship  between  central  and  local 
use  of  information  systems,  and  the  provision  of  citizen 
services,  together  with  the  accomplishment  of  social  goals) . 
Again,  academic  specialties  were  not  listed  in  all  cases, 
hence  the  unknowns. 
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